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ABSTRACT 

Opinions on undergraduate psychology programs were 
obtained from 348 psychology alumni who graduated between 1978 and 
1988, and from 101 psychology faculty who taught during this same 
period on eight campuses of California state University. Twenty items 
reflecting student development and eight items measuring program 
quality were factor analyzed separately and were combined into five 
factors (personal growth, cultural diversity, basic skills, 
scientific principles, and program quality). Alumni were generally 
positive about their programs and their learning. Alumni were 
particularly aware of the major's contributions to their personal 
growth. Faculty mean responses about student personal growth and 
learning were generally between '^fair" and "good," with hign3r 
ratings given to basic skills and scientific principles items. Alumni 
and faculty mean ratings were significantly different overall on 4 of 
the 5 factors and on 21 of 27 items. Every item assessing student 
development that revealed a signifj^cant difference was marked higher 
by alumni, and every item assessing program quality was marked higher 
by faculty. (Six references) (JDD) 



* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



Alumni vs. Faculty Opinion 

1 



Ali;mni vs. Faculty Opinion on Undergraduate Psychology Programs 
Mary J. Allen, Colin A. Armstrong, and Debbi M. Gutierrez 
California State University, Bakersfield 

Paper presented at the Annual Convention of the American 
Psychological Association, Boston. August, 1990 



Opinions on undergraduate psychology programs were obtained from 
348 psychology alumni who graduated between 1978 and 1988 and 
101 psychology faculty who taught during this same period on eight 
university campuses. Twenty items reflecting student development 
and eight items measuring program quality were factor analyzed 
separately and were combined into five factors. Correlational 
results showed a general consensus between alum.ni and faculty on 
the relative achievements of students, but discriminant analysis 
revealed mean differences, with alumni reporting greater growth and 
development and faculty rating aspects of the program, such as the 
quality of advising, more highly. 
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Alumni vs. Faculty Opinion on Undergraduate Psychology Programs 

Undergraduate psychology education is thriving. In 1987 alone, 
over 42,000 psychology baccalaureate degrees were awarded 
(National Center for Education Statistics, 1989). Psychology faculty 
need to examine their programs and evaluate how well the goals of 
this enormous enterprise are being accomplished. 

The empirical focus and psychometric background shared by 
psychologists has prompted repeated assessments of their alumni. 
After reviewing the results of such surveys, McGovern and Carr 
(1989) recommend ongoing alumni surveys, with results directly 
linked to program evaluation and change. Halpern's (1988) view is 
similar, stressing that alumni surveys should be used to evaluate 
progress toward locally determined goals, a process requiring 
considerable faculty involvement. 

Student outcomes assessment is not restricted to psychology 
and is not always conducted by faculty for faculty. Taxpayers, 
legislators, and private funding sources have Increasingly begun to 
demand accountability for the expenses of supporting higher 
education (Halpern, 1988). The question has long passed from one of 
whether or not to undertake student outcomes assessment to one of 
which assessment method is most effective (Chandler, 1987). 

A variety of assessment techniques are available. Traditional 
measures can be adopted from a "management perspective" (Kozloff, 
1987, p. 420), one which gauges success by numbers of graduates 
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and job placements or resource counts (e.g., library size, computer 
availability). Other measures involve using standardized exams to 
evaluate student learning. These approaches, when used alone, 
involve narrow definitions of program excellence which have the 
potential to substitute quantity for quality, to discourage student 
diversity, and to encourage "teaching to the test" phenomena. As 
Halpern (1988) points out, many measures of institutional quality 
(e.g., campus size, library contents, prestige of funding sources) 
have little relationship to what students find of critical value: how 
much they have learned. This study examines alumni opinions 
concerning program quality and what they have learned, attitudinal 
variables that should not be ignored. 

One danger of obtaining alumni opinions is that faculty may be 
tempted to conduct post hoc analyses, perhaps displaying some of 
the "l-knew-it-all-along" phenomenon (Myers, 1989, p. 20). tn this 
study, faculty and alumni were simultaneously and independently 
surveyed on their opinions about program quality and contributions 
to student development. This approach provides the opportunity to 
isolate both consensus and divergence of opinion. In addition, this 
survey was conducted at eight different California State University 
(CSU) campuses, each with an independent curriculum and faculty, so 
comparisons of alumni and faculty opinions can be examined in the 
aggregate and at separate universities, allowing for assessment of 
the generalizability of results. 

Method 
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Opinions on the undergraduate psychology program were 
obtained from 348 psychology alumni who graduated between 1978 
and 1988 and 101 psychology faculty who taught during that period 
at eight CSU campuses. The CSU system consists of 20 campuses 
and currently enrolls over 365,000 students, and the eight selected 
campuses represent all major geographical areas in the state. 
Contacts on each of these campuses, who were guaranteed campus 
and respondent confidentiality, generated a list of names and 
addresses of all known alumni and all present and former faculty. 
From 35 to 55 alumni and from 4 to 20 faculty from each campus 
responded to the survey. 

For the alumni, random or universal samples were drawn from 
each campus, depending on population sizes, and alumni were 
contacted at least twice, first with a cover letter and 
questionnaire, then with a follow-up reminder postcard. If fewer 
than 50 questionnaires from each campus were returned, either a 
third mailing was made to non-respondents, or, if the list contained 
more possible respondents, a replacement sample was drawn and 
contacted. Universal samples were selected for faculty, who were 
contacted with a cover letter and questionnaire. Faculty who did not 
respond to this first mailing were sent a foilow-up postcard, and, 
finally, an additional letter stressing the importance of their 
response. The estimated return rate is 28% fc- alumni and 34% for 
faculty. (These estimates are conservative since the number of 
letters that were not received because of address and name changes 
could not be calculated). 
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Both faculty and alumni questionnaires were tested on a pilot 
campus. Each contained four pages of questions that collected 
demographic information, ratings of contributions of the major to 
student development, and ratings of aspects of program quality. 

For the alumni, the contributions of the major to student 
development were assessed by a list of 20 Items preceded by the 
following question: "How well did your undergraduate major add to 
your development in the following areas?" (For faculty this wording 
was changed to "How well did the major contribute to student 
development in the following areas?") Each item had four-point 
Likert scale response categories, ranging from "Poor" (1) to 
"Excellent" (4). A "Not Applicable" rating was also available. An 
additional set of six items assessed the quality of the program using 
the same scale and the following instructions: "Again, thinking 
about your undergraduate major , please rate the following for the 
campus from which you graduated." (Correspondingly, faculty were 
asked to "Please rate the following aspects of this program.") 

The following question was asked of alumni: "When you reflect 
upon your time at this campus, how often did the faculty in your 
major challenge you to do the very best you could do?" For faculty, 
the question was phrased "When vou reflect on your teaching 
experience at this campus, did you challenge stJjdents to do the very 
best they could do?" For both the alumni and faculty, possible 
responses ranged from "Never" (0) to "Always" (4). 

Alumni were asked ' i/Vhich of the following best represents how 
you feel about your A.B. degree?", while faculty were asked "Which 
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of the following best represents your view of the undergraduate 
degree program in which you taught at your campus?" Four possible 
responses were provided, ranging from "It is a dugree of poor 
quality." (1) to "It is a degree of excellent quality" (4), 

The alumni were 71% female and 86% white, with a mean age of 
33.07 (SII=8.64) and estimated mean age at graduation of 27.97 
(SD«.8.23). The faculty were 38% female and 100% white, with a 
mean age of 50.39 (SI1= 10.20). 

Results 

Twenty student development items were factor analyzed and 
yielded four factors: Personal Growth, Cultural Diversity, Basic 
Skills, and Scientific Principles. (One of these items loaded on 
multiple factors and was excluded from subsequent analyses). The 
six program quality items, the "challenged" item, and the "quality of 
the degree" item were factor analyzed and yielded one factor: 
Program Quality, Five factor scores were created by averaging 
scores for the relevant variables, and, to reduce the effects of 
missing data, factor scores were created if no more than one 
relevant Item was omitted. Table 1 lists the items contributing to 
each factor. 

A discriminant analysis comparing alumni to faculty on the five 
factors was significant, (5, a=3B1) = 170.91, q, < .0001, with a 
canonical correlation of .60. Eighty-two per cent of the alumni and 
85% of the faculty were correctly classified usmg the discriminant 
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function. Univariate follow-up t-tests were performed on the 
factors and individual items. Results are summarized in Table 1. A 
parallel set of discriminant analyses comparing alumni to faculty at 
each of the seven campuses having at least eight faculty 
respondents were all significant at c.< 001, with canonical 
correlations ranging from .56 to .78, with a median of .59. Correct 
classifications ranged from 73 to 91% (with a median of 84%) for 
alumni and from 70% to 100% (with a median of 89%) for faculty. 

There was a significant Spearman correlation between alumni 
and faculty mean ratings of the 19 student development items, is = 
.70, p. » .001. The Spearman correlation between the alumni and 
faculty mean ratings of the 8 program quality items was also 
significant, is = -69, q, < .05. Parallel correlational analyses were 
performed for each of the seven campuses with at least eight 
faculty respondents. The correlations between alumni and faculty 
mean ratings of student development items were all significant at 
a<.05 and ranged from .36 to .78, with a median of .62. Three of the 
correlations between alumni and faculty mean ratings of program 
quality items were significant at a<.05 and ranged from .77 to .90. 
The non-significant correlations ranged from -.13 to .34. 

Discussion 

Data were analyzed in two different ways, examining mean 
differences and correlations between means. While mean 
differences between alumni and faculty were apparent, correlational 
results indicate considerable agreement between the two groups on 
what the psychology major is accomplishing. 
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Alumni were generally positive about their programs and their 
learning. High averages suggest that alumni are particularly aware 
of the major's contributions to their personal growth, especially 
adaptability, interest in lifelong learning, and self- understanding. 
Alumni ratings of the cultural diversity items tended to be lower 
than their ratings of personal growth items, but averages generally 
were still closer to 3.0 ("Good") than 2.0 {"Fair"). Their ratings of 
basic skills and scientific principles items were about 3.0 ("Good"), 
w.^h highest ratings associated with learning the content of the 
major. Alumni mean responses on the program quality items are 
generally nea^ "Good," ranging from "Fair" on preparation for 
employment to "Good" for accessibility of faculty. 

Faculty mean responses about student personal growth and 
learning were generally between "Fair" and "Good", with higher 
ratings given to basic skills and scientific principles items. 
Program contributions to planning projects, understanding written 
material, learning scientific methodology, statistics, and knowledge 
within the major earned higher ratings from faculty. All but one 
item within the Program Quality factor were rated between "Good" 
and "Excellent" by faculty. The highest ratings were for quality and 
availability of courses and preparation for graduate school. The only 
item rated below "Good" was preparation for employment. 

Alumni and faculty mean ratings were significantly different 
overall, on four of the five factors, and on 21 of the 27 items. The 
pattern was simple and consistent. Every item assessing student 
development that revealed a significant difference was marked 
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higher by alumni, and every item assessing program quality was 
marked higher by faculty. From the faculty's perspective, alumni 
overestimated how much they learned; from the alumni's 
perspective, faculty overestimated the quality of the program. 

Mean differences were especially consistent for two of the four 
student development factors: Personal Growth and Cultural 
Diversity. Alumni and faculty had significant mean differences on 
every item for these two factors. Alumni ratings on the Personal 
Growth factor averaged a full half-point higher than faculty ratings, 
almost a whole standard deviation. They particularly disagreed on 
the major's contribution to lifelong learning, leadership skills, and 
identifying values. 

Each of the eight campuses in this study operates independently 
and has its unique student population, faculty, and program, allowing 
for an examination of the generalizability of results. Alumni and 
faculty rank orderings of program quality items at different 
campuses do not consistently agree. However, the majority of eight 
campuses showed identical patterns of alumni/faculty mean 
differences and agreement betv/sen alumni and faculty on the 
relative attainment of aspects of student learning, suggesting that 
these findings are generalizable to other universities. 

When interpreting alumni surveys, faculty should not be overly 
disappointed by weaker than expected alumni ratings of general 
aspects of program quality; neither should they feel uniquely 
effective if alumni give more positive evaluations of their own 
learning than expected. There is considerable ac eement between 
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faculty and alun nl on the relative accomplishments in student 
development. The next question is one of "What should we be 
accomplishing?", rather than "What is being accomplished?" 
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Table 1 Ratings of Aspects of the Undergraduate Major by Alumni and Faculty 
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Personal Growth (7) 
Being Independent (7) 
Being Adaptable (6) 
Understanding Self (7) 
Identifying Values (7) 
Lifelong Learning (7) 
Group Cooperation (7) 
Leading Others (7) 

Cultural Diversity (7) 
Gender Roles (6) 
U. S. Minorities (6) 
World Cultures (7) 

Basic Skills (5) 

Planning Projects (4) 
Writing Well (5) 
Speaking Well (6) 
Understanding Writing (5) 

ScienUflc Principles (4) 
Logical Inferences (6) 
Scientific Methods (4) 
Statistics/Math (4) 
Technology Impact (6) 
Content of Major (5) 

Program Quality (7) 
Accessibility of Faculty (5) 
Helpfulness of Advisor (7) 
Quality of Courses (7) 
Availability of Courses (7) 
Preparation for Employment (7) 
Preparation for Grad School (7) 
Challenge by Faculty (7) 
Quality of Degree (7) 

NQt£. Numbers in parenthesis indicate the number of campuses that show the 
same pattern of mean differences between alumni and faculty. 
*p<.05. ♦♦p<.01. ♦♦♦p<.005. ****p<.001. 
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